Overcoming the three-item limit: Gestalt grouping principles explain increases in change detection capacity

Previous work suggests that visual processing includes an object-based short-term memory that can represent information from approximately three objects in parallel. Five experiments using a change detection task with grids of simple items varying only in luminance explored what counts as an object for visual short-term memory and how visual long-term memory might contribute to performance. Experiments 1 and 2 revealed a capacity limit of approximately three objects for vSTM which is consistent with estimates from other paradigms. This limit was robust to changes in display time and array size. Experiments 3-5 examined the effect of long-term exposure to an array on change detection performance and found no significant improvement relative to new arrays, suggesting that the three-item limit is a fixed structural limit. Moreover, an algorithm based on the classic Gestalt principles of similarity and proximity accounted for any apparent instances of capacity greater than three items. Although perceptual grouping can change what counts as an “object” for vSTM, the three-item limit of vSTM appears impenetrable to the ameliorating effects of familiarity. Introduction When viewing a scene, observers shift attention from one region to another, encoding selected aspects of the scene into visual memory. Given that the capacity of visual memory appears limited, recent research has focused on the ways in which it is limited. How much of a display can we attend to at once and how much of that information is preserved in visual short-term memory (vSTM)? If attention focuses on discrete objects rather than spatial regions of a scene (Kahneman, Treisman & Gibbs, 1992; Luck & Vogel, 1997), and if encoding into vSTM requires focal attention, then the limits on visual memory will depend on the number of objects that can be encoded in a single attentional “glance” — how many objects can be attended to in parallel. Converging evidence from a wide variety of tasks suggests a magic number of approximately 3-4 items as the capacity limit on attention and vSTM (Luck & Vogel, 1997; Pashler, 1988; Phillips, 1974; Pylyshyn & Storm, 1988; Scholl & Xu, 2001; Sperling, 1960; Vogel, Woodman & Luck, 2001; for review see Scholl, 2001). For example, adults can use attention to keep track of the motions of up to four identical, randomly moving items and distinguish them from four identical distractor items (Pylyshyn & Storm, 1988; Scholl & Pylyshyn, 1999). Both human infants and non-human primates show similar limits in tracking multiple objects; 10 to 12-monthold infants can track up to three objects (Feigenson, Carey & Hauser, 2002; Feigenson & Carey, 2003) and rhesus monkeys can track up to four objects (Hauser, Carey & Hauser, 2000; Hauser & Carey, 2003). This magic number extends to other attention and memory tasks. For example, transsaccadic memory is limited to about four items (Irwin & Gordon, 1998), adults can detect changes to the properties of about four objects in parallel (Luck & Vogel, 1997), and approximately four items can capture attention in a visual search task (Yantis & Johnson, 1990; but evidence from visual marking tasks might suggest a higher number, see Belopolksy, Theeuwes, & Kramer, in press). Despite the consistency of evidence for a limit of 3-4 items, a few tasks suggest lower or higher limits. The measured capacity for complex stimuli is often fewer than three items (Phillips & Christie, 1977; Alvarez & Cavanagh, 2003), suggesting an additional limit on the amount of information that can be retained. With complex objects, the information limit is exceeded before the limit of 3-4 objects is reached (Alvarez & Cavanagh, 2003). In contrast, with arrays of simple objects, capacity estimates can be substantially higher than 3-4 objects (Rensink, 2000a). This increased capacity might well result from grouping or chunking processes that allow more information to be encoded into a single higher-order object. For example, three adjacent, similar items might be treated as a single object, resulting in a capacity limit greater than four individual items, but not necessarily more than four “objects” (see Cowan, 2000 for a related discussion of chunking in verbal short-term memory, and see Feigenson & Halberda, 2004 for evidence of visual chunking by infants). In this paper, we use a change detection task to explore the effects of grouping on estimates of the capacity of visual memory and attention. In the flicker change detection task, original and modified displays alternate, separated by a brief blank screen, until subjects find the change (Rensink, O’Regan & Clark, 1995; Rensink, O’Regan & Clark, 1997; Rensink, 2000). With changes to photographs of natural scenes, observers often need many alternations in order to detect large changes, a phenomenon known as change blindness (Rensink et al, 1997; Simons & Levin, 1997). Change detection tasks provide a useful tool for the study of attention and capacity limits because attention is thought to be necessary for change detection (Rensink et al, 2000; Rensink et al, 1997). Successful change detection requires subjects to encode the pre-change display and then to compare their representation to the post-change display (Simons, 2000). If subjects could attend to and successfully encode all of the elements in a display in parallel, then they potentially could detect changes with a single exposure to the change. However, if the number of objects in the display exceeds the capacity to encode and remember them or if the display duration is too brief to complete encoding, observers will require multiple cycles to detect a change. If observers encode as many objects as possible into visual memory, then given sufficient time, they will maximize the contents of visual memory, filling it to capacity. Once the capacity limit is reached, additional exposure time will not enhance change detection because no additional items could be encoded. This asymptote of change detection performance as a function of display exposure time allows an assessment of capacity limits on the encoding of the displays. Rensink and colleagues (2000) used this approach with simple arrays of objects to explore the capacity of visual memory for basic feature properties such as orientation and polarity. With a display time of approximately 600ms, search efficiency for an orientation change asymptoted, leading to an estimated capacity of 5.5 items (the method for estimating capacity is described more fully in the results section). In contrast, polarity changes never asymptoted with display durations up to 800ms, leading to a minimum capacity estimate of 9 items. Both of these values exceed the more typical 3-4 item limit for visual short term memory, raising the intriguing question of why subjects showed superior memory for these simple object properties. One plausible explanation is that subjects grouped items together as if they were parts of a single object, and this grouping led to inflated estimates of the number of items that could be retained and compared. One central goal of our research is to address the nature of these grouping mechanisms and their influence on estimates of capacity. Because change detection requires the subject to compare the contents of visual short-term memory (vSTM) to a present visual scene (i.e. the 1 display to the 2 display), this task does not distinguish between a capacity limit of vSTM itself and a capacity-limited comparison process. The empirical literature often collapses across these two potentially separable components of midlevel vision (see Rensink 2000c or Mitroff & Simons, in press for discussion). vSTM appears to be an object-limited store that encodes information in object-based rather than retinotopic coordinates (Phillips, 1974), but an object-based attention mechanism involved in the comparison process may also constrain performance. Although we remain agnostic about the relative contributions of these potential limits to estimates of 3-4 item capacity in change detection tasks, throughout this paper, we describe the limits as constraints on vSTM. Another concern with estimates of capacity based on the flicker task is that long exposure times might allow subjects to rely on long-term rather than short-term memory to perform the task. Increasing the initial exposure time for natural scenes does not greatly facilitate change detection in the flicker task (Rensink, O’Regan, & Clark, 2000), but other evidence suggests that some information from the pre-change display can be encoded into long-term memory (Simons, Chabris, Schnur, & Levin, 2002) and that long-term memory may contribute to change detection (Hollingworth, Williams, & Henderson, 2001). If long-term memory can be used for change detection in the flicker task, it might spuriously inflate estimates of the capacity of vSTM. In several of our experiments, we systematically manipulate familiarity with the displays in order to determine whether long-term memory increases capacity estimates in a change detection task. In Experiment 3, we compared performance when a search display was repeated on many trials to performance when a different search display appeared on each trial. In Experiments 4 and 5, two observers memorized a display to a level that far exceeded the specificity required for change detection. We then estimated how this precise long-term memory influenced performance on a change detection task. Finally, we examined the extent to which grouping contributes to capacity estimates, both with and without contributions from long-term familiarity with the displays. In sum, this paper examines: (a) whether the capacity of visual short-term memory and attention can be increased beyond 3-4 items via contributions from long-term memory, and (b) whether the effects of familiarity result from an increase in the number of items that can be retained in vSTM or from grouping more information into each memory item. Unlike earlier change detection studies, we also used displays that could not be easily encoded categorically or verbally. Consequently, capacity estimates should be more closely tied to visual encoding than to other, non-visual memory